Data Mining Tools Make Flights Safer, More Efficient 


NASA Technology 

W hen an aircraft experiences catastrophic fail- 
ure, it’s not difficult to tell that something has 
gone wrong. But even if an airplane reaches 
its destination without an incident, it may still have bent 
or broken the boundaries of safe operating procedures. 
Figuring out what those boundaries should be — what 
“normal” operation should look like — and devising ways 
of effectively tracking them can be a tough task. Typically, 
this is accomplished using modeling and real-world tests, 
followed by equipping vehicles with a variety of sensors so 
that flight data can be recorded and analyzed. 

But an average commercial airplane generates a 
massive amount of data each flight: everything from 


instrument positions to sensors to voice recordings are 
collected and stored for later analysis. With so much data, 
there is no good way for problems to stand out unless a 
commercial company already knows what those problems 
look like. Says Captain Jeff Hamlett, the director of flight 
safety at Southwest Airlines Co., “We have mounds of 
data; the big request is always, ‘Tell me something I don’t 
know.’ We have to start with something specific, like an 
issue we discovered in a pilot report, and then we can 
search through the system and discover the breadth and 
depth of the issue.” 

Discovering these concerns and addressing them 
before they cause an accident is also one of NASA’s 
missions. The Aviation Safety Program under the 




Tower controllers test out NASA software 
tools in a simulation at the Agency’s Future 
Flight Central air traffic control tower 
simulator at Ames Research Center. 


Aeronautics Research Mission Directorate has many 
initiatives to enable safer air transportation systems. 
Among them, in the Intelligent Systems Division at Ames 
Research Center, is a small team of researchers who have 
been tasked with applying data mining algorithms to avia- 
tion safety problems. 

Technology Transfer 

NASA has lots of expertise in data mining — that is, 
creating tools to discover interesting patterns in large data 
sets. But in order to ensure its techniques can make a real 
impact on flight safety and efficiency, Ames has sought 
the help of commercial partners, among them Dallas- 
based Southwest. 

Says Nikunj Oza, a researcher at Ames and leader of 
the data mining team, “We’ve made partnerships with 
Southwest and others, who then make their data acces- 
sible to us to facilitate our development. In return, they 
get certain insights into their data, and they can give us 
feedback on how well our algorithms are working, as well 
as what adjustments might make them more accurate or 
user friendly.” 

For Hamlett, the value of a NASA-Southwest 
partnership had become apparent in 2003 , when 
Southwest identified a higher rate of steep approaches in 
the flight data than was being reported by its pilots. “We 
turned to our pilot self-reporting program, expecting to 
find lots of reporting about unstable approaches, but we 
couldn’t match the number that we were seeing in our 
data,” he says. 

Hamlett instead tried searching by phrases within pilot 
reports, using a NASA-developed tool called Perilog. “It 
turns out that when you read some of the elements of 
the report,” he says, “they are describing an unstable 
approach — saying things like ‘we were struggling to get 
the airplane configured’ or ‘we were too fast’ — and using 
all the words that describe an unstable approach, even if 
they hadn’t reported it as such.” Southwest used reported 
information to identify and communicate certain issues 
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to air traffic controllers, making them aware, for example, 
of how certain instructions impacted an airplane’s opera- 
tions. By collaborating with them, Southwest has since 
seen a steady improvement in the quality of approaches. 

“That’s just one example of how NASA provided us 
a tool that helped us look at our data in a way we never 
thought about before,” says Hamlett. 

Another collaborative effort between Southwest and 
the Human Factors Division at Ames resulted in an 
improved, rewritten set of operating procedures for the 
airline in 2004. Says Hamlett, “They played a big part in 
making sure that we get a good process for reviewing all 
of our normal operations.” And from that work, Hamlett 
made the acquaintance of Ashok Srivastava, then head of 
the data mining team in the Aviation Safety Program at 
NASA. 

Following year-long discussions between Hamlett 
and Srivastava, in 2011 NASA and Southwest signed a 
Space Act Agreement to share algorithms and 
flight data. The arrangement would 
help the Ames team ensure that 
their research was suitable for 
real-world applications, and 
it would help Southwest to 
make the most out of its large 
and growing body of data. 

Benefits 

As Oza puts it, NASA’s algorithms — 
the two used by Southwest are the Multivariate 
Time Series (MTS) search and Virtual Sensors — 
specialize in “letting the data speak for itself by 
finding unusual flights and candidate anomalies, with- 
out having any preconceived notion of what is normal 
or abnormal. The drawback is that sometimes you find 
statistical anomalies that are not safety concerns, but 
the benefit is that sometimes you’ll find anomalies that 
you weren’t seeing before that turn out to be safety 
concerns or have other operational significance, such 
as excessive fuel use.” 

One such revelation came when Southwest began 
tracking planes in an effort to boost fuel efficiency. 
Says Hamlett, “What we were trying to do was look 


at the performance of aircraft between certain cities 
to see if there was a particular plane or city pair that 
was off normal and causing us to burn too much fuel.” 
What Southwest discovered instead was an unusual 
signature with one particular plane, which when inves- 
tigated turned out to be caused by inaccurate sensors. 
“It wasn’t what we set out to do, but it turned into a 
success for us,” says Hamlett. 

Like many airlines, Southwest has traditionally 
looked for performance issues in its data using exceed- 
ances checked against a model. During arrival, for 
instance, the plane might record an exceedance if it is 
travelling faster than 250 knots while its altitude is less 
than 10,000 feet. Each morning, the company looks 
at a report of all the exceedances that took place the 





previous day and decides what action to take in the 
case of undesirable trends. 

Using NASA’s tools, Hamlett says Southwest can 
now query the data itself to figure out what normal 
operations really look like. “Just because we expect an 
airplane to be on speed at 1,000 feet, on glide path, on 
course, doesn’t mean that’s what normally happens. I 
think what we got out of this technology is the ability 
to ask, ‘What is normal?’ Because that turns out to be 
really powerful, and we can say ‘OK, we need to make 
this correction in our training,’ or ‘Maybe we need to 
adjust our concept of what the ideal is.’” 

Oza is glad to see that NASA’s work is making an 
impact on commercial flight safety. “If you go into 
a dark room with a flashlight, your beam only hits a 
small area, and you don’t know what’s in the space 
that’s dark. We’re essentially going in with a broader 
beam, if not a floodlight. What you find may or may 
not turn out to be problematic — but at least you 
now know what’s there.” ♦♦♦ 
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Using algorithms created by NASA, Southwest Airlines was able to 
identify areas of concern in certain flight data. By working with its 
pilots and air traffic control, the company has since seen an increase 
in the number of stable approaches by its aircraft. 
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